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Image-Capture Event Monitoring 

Field of the Invention 

5 The present invention relates to image-capture event monitoring. 

As used herein, the term "image capture" is intended to cover any form of image capture by 
camera functionality whether involving still photographs or video or film clips and 
including both visible-light image capture and infra-red image capture. 

10 

Background of the Invention 

It is a common human experience to arrive at a seemingly interesting attraction, such as 
film or television star making a public appearance or a Mickey Mouse character giving out 
toys to children in a theme park, just as the attraction has finished and the crowds are 
15 dispersing. Many interesting attractions and places to visit are missed by most people 
simply because they are unaware of that these attractions and places exist; furthermore, 
often such attractions and places are not to be found in the standard guides. 

It is an object of the present invention to facilitate the detection of interesting events and 
20 places. 

The present invention has as one of its foundations the simple observation that image- 
capture activity by people is a good indicator of the presence of an attraction or place of 
interest, at least to a certain class of people. By monitoring and analysing the image-capture 
25 activity of people, it is possible to derive useful information about attractions and places. 

It is known from our published European specification EP 1 128 284 A to transmit the 
location of an image capture event, without the corresponding image itself, to an internet 
service system for temporary storage. This specification also discloses a location-based 
30 electronic photo album in which images thumbnails can be displayed on a map in a manner 
indicating where the corresponding images were captured. Where there are too many 
images associated with a particular locality to show at a current map resolution, the images 
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thumbnails are replaced by an icon indicating that a cluster of images is associated with the 
locality concerned. 



Summary of the Invention 

5 According to one aspect of the present invention, there is provided an image-capture event 
monitoring method comprising the steps of : 

(a) receiving, from multiple different sources, image-capture event notifications and 
deriving from each notification a location parameter indicative of where an image- 
capture event has occurred; 
10 (b) using said location parameters to associate image-capture events into one or more 
clusters; and 

(c) analysing a said cluster of image-capture events in dependence on at least one further 
parameter of each event. 

15 In one preferred embodiment, the said at least one further parameter is the time of 
occurrence of each image-capture event; in this case, the analysis effected in step (c) can 
advantageously be used to detect rapid increases in image-capture activity indicative of a 
currently-happening attraction, or to detect periodicity in the image-capture events 
indicative of the regular times of occurrence of an attraction. 

20 

In another preferred embodiment, the said at least one further parameter is the direction of 
image capture associated with each event; in this case, the analysis effected in step (c) can 
advantageously be used to determine a subject of the image-capture activity or a 
characteristic of the locality where the images are being captured. 

25 

According to another aspect of the present invention, there is provided a service system for 
monitoring image-capture event, the system comprising: 

- an input interface for receiving, from multiple different sources, image-capture event 
notifications and for deriving from each notification a location parameter indicative of 

30 where an image-capture event has occurred; 

- a data store for storing data derived from said event notifications; 



- a first processing arrangement for using said location parameters to associate image- 
capture events into one or more clusters; and 

- a second processing arrangement for analysing a said cluster of image-capture events in 
dependence on at least one further parameter of each event. 



Brief Description of the Drawings 

Embodiments of the invention will now be described, by way of non- limiting example, 
with reference to the accompanying diagrammatic drawings, in which: 
. Figure 1 is a diagram of a service system embodying the invention; 
1 0 • Figure 2 is a diagram illustrating the clustering of image-capture events by location; 
. Figure 3 is a diagram showing the Figure 2 event clusters overlaid on a map; 
. Figure 4 is a diagram illustrating how the direction of image capture of events of a 

cluster enable the likely subject of the image-capture events to be 

determined; 

1 5 . Figure 5 is a diagram similar to that of Figure 3 but for a different cluster of events; 
. Figure 6 is a diagram illustrating how the direction of image capture of events of a 

cluster enable a characteristic of the locality to be determined; 
. Figure 7 is a diagram illustrating the detection of a currently-occurring attraction by 
monitoring the rate of accretion of events to a cluster; 
20 . Figure 8 is a diagram illustrating a periodicity in the occurrence of image-capture 
events of a cluster; and 
. Figure 9 is a diagram illustrating the expected progress of a current occurrence of a 
regular attraction where the current occurrence has been detected by the 
rate of accretion of image-capture events related to the attraction exceeding 
25 a threshold. 



Best Mode of Carrying Out the Invention 

Figure 1 depicts a "photo-opportunity" service system 14 connected to a communications 
infrastructure 1 2 in order to receive image-capture event notifications 1 3 and to respond to 
30 user requests 19. 
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The image-capture event notifications 13 serve to notify the service system 14 of 
corresponding image capture events effected by camera functionality 10, the notifications 
being sent to the service system 14 by communications functionality 11. The camera 
functionality 10 can be provided in stand-alone apparatus or incorporated along with other 
5 functionality into multi-function apparatus. In particular, the camera functionality and 
communications functionality 1 1 may be combined into a single apparatus such as a 
mobile phone/camera combination. 

The image-capture event notifications are preferably sent immediately after the 
10 corresponding image capture events have occurred but it is also possible to delay sending 
notifications to enable, for example, a batch of notifications concerning the image-capture 
event of a single camera to be sent all together (typically at a fixed time or when an image- 
recording medium of the camera is full). Thus, in one scenario involving a digital camera 
with a memory stick for holding the captured images, the image-capture event notifications 
15 can be arranged to be sent when the contents of the memory stick are uploaded into 
equipment such as a home computer - in this case, the communications functionality 1 1 is, 
for example, constituted by internet connectivity functionality of the computer, the 
notifications 13 being sent out by the computer over the internet to the service system 14. 

20 The communications infrastructure 12 can be of any suitable form and will typically 
comprise multiple interconnected networks of various types. For example, the service 
system 14 can be connected to the internet with the communications functionality 1 1 being 
arranged to communicate with the internet via a wireless LAN (such as an 802.11 
network), a mobile phone network, a PSTN network, a wired LAN, or any suitable 

25 combination of the latter. 



The service system 14 is arranged to receive image-capture event notifications 13 from 
multiple users in order to gather information about attractions and places of interest from 
multiple different sources rather than relying on the experiences of a single person. 

30 

The service system 14 is further arranged to service requests 19 from users (typically, 
subscribers to the service offered by the system 1 4), these requests being generated by user 
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interface apparatuses 25. Whilst the individual apparatuses 25 and their users can be 
completely independent of the apparatuses providing cameras functionality 1 0 and their 
users, it is envisaged that often they will involve the same items of apparatus and users. As 
will be more fully explained hereinafter, in the present embodiment two types of request 19 
5 are provided for, namely a request for real-time information (that is, a request wanting an 
immediate response), and a request to be sent an alert upon certain trigger conditions being 
met. 

Each image-capture event notification 13 comprises first data concerning a location 
10 parameter indicative of the location of occurrence of the corresponding image-capture 
event; the notification 13 may also comprise second data relating to one or more further 
parameters of the event, such as its time of occurrence and/or the direction of image 
capture and/or an ED of the camera functionality or the associated user. 

15 The first data can directly comprise the location of the image-capture event, or data 
enabling this location to be retrieved from a location server. In Figure 1, dashed box 20 
indicates location discovery functionality for providing the location of the camera 
functionality 10 at the time an image-capture event takes place, the capturing of an image 
triggering a request to the functionality 20 for a location fix. By way of example, three 

20 location discovery techniques are indicated in box 20, namely: 

- the use of a GPS system (typically incorporated in the same apparatus as the camera 
functionality 10); 

- the use of location beacons that transmit their respective locations using short-range 
communication links such as infra-red or Bluetooth wireless links (in this case, the 

25 apparatus providing the camera functionality 10 is equipped with an appropriate 
receiver); 

- the use of a mobile phone network (Public Land Mobile Network, or PLMN) to locate 
mobile phone functionality associated with the camera functionality 10. 

It will be appreciated that any other suitable location discovery technique can be employed. 
30 Where the location of the camera functionality 10 is provided using a PLMN, the location 
information can either be provided to the camera functionality 10 or communications 
functionality 1 1 directly (see arrow 22) to enable the location to be included as the first 
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data in the image-capture event notification, or else the location information can be stored 
in a location server 21 accessible via the communications infrastructure 12. In this latter 
case, the first data comprises identification data for use in retrieving the location 
information from location server 21, the retrieval being effected (see arrow 23) by the 
5 service system 12 upon receipt of the event notification; the identification data may, for 
example, comprise a user identifier and password, and a timestamp or other element for 
associating the image-capture event concerned with the correct item of location 
information held by the location server. 

10 The service system 14 comprises an input interface 15 for receiving the image-capture 
event notifications 13, a database 16 for storing information directly or indirectly derived 
from the notifications 13, a processing sub-system 17, and a request handler 18 for 
handling requests from user interface apparatuses 25. 

1 5 The input interface 1 5 is operative upon receiving an image-capture event notification 1 3 
to derive the location parameter of the corresponding event either simply by extracting it 
from the notification or by using identification data included in the notification to retrieve 
the location parameter from the location server 21. The location parameter of the event, 
together with any further event parameters included in the notification 13, are stored (at 

20 least temporarily) in database 1 6 against a unique event ID; thereafter the processing sub- 
system 17 is informed of the newly-notified event. 

The processing sub-system 1 7 (which is typically a program-controlled processor) provides 
cluster functionality 17A for associating events by their locations to form one or more 

25 event clusters. Upon the sub-system being informed of a newly-notified event, the 
functionality 17A determines on the basis of the location parameter of the event whether 
the event qualifies as a new member of an existing cluster of events and if this is not the 
case, whether the event can be used along with one or more previously-notified events not 
associated with any cluster, to form a new cluster. Any cluster membership determined for 

30 the newly-notified event is then stored in database 16 against the event ID. As for the 
criteria to be applied to determine whether an event qualifies as belonging to a particular 
cluster, a number of clustering algorithms are well known in the art and can be adapted for 
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use in the current case. Indeed, any suitable criteria can be applied such as the proximity of 
the two nearest events to the event under consideration both being below a particular 
threshold. 

5 The clustering functionality may rely on information additional to event location in making 
its cluster determinations. For example, reference to a map may show that events that are 
closely located are separated by a barrier that make it likely that events on opposite sides of 
the barrier are not really associated and should be treated as belonging to different clusters. 
A similar determination may also be made in the case where the direction of image capture 
1 0 of events is known and there appear to be two distinct image-capture directions associated 
with closely located events. 

The processing sub-system 17 also includes analysis functionality 17B for analysing each 
cluster of events against one or more further event-specific parameters such as the time of 
15 occurrence, and/or direction of image capture, and/or source ID of each member events. 
The frequency of analysis depends on the purpose of the analysis; for example, it maybe 
appropriate to carry out a new analysis of a cluster each time a new member is added or 
simply to do the analysis at fixed intervals. 

20 The results of the analyses carried out by analysis functionality 1 7B are stored in database 
16 to be available to the request handler 18 for responding to requests 19. In certain cases, 
an analysis may determine that an alert should be generated; in this case, the request 
handler 1 8 is directly informed to enable it to immediately send the alert to any user who 
has requested to receive the alert concerned. 

25 

The requests 19 will typically be location-based requests for information concerning 
attractions and places of interest near to the requestor (user of apparatus 25). The location 
of the requesting apparatus 25 is, for example, derived in a manner similar to that described 
above in respect of the location of an image-capture event; furthermore, like the location of 
30 an image-capture event, the location of requesting apparatus maybe provided to the service 
system in the request itself or the latter may incorporate identification data for enabling the 
locatin to be retrieved from a location server. Where the request relates to an alert, since 



8 

the alert may not be generated for some time, the request handler must either be provided 
with regular location updates concerning the requesting apparatus, or else the request 
handler must check the location of all apparatuses that have requested alerts whenever an 
alert is generated by the processing sub-system 17. 

5 

Having described the general form and functionality of the Figure 1 embodiment, a more 
detailed description will now be given of certain aspects and, in particular of the processing 
carried out by the processing sub-system 17. 

1 0 Figure 2 illustrates an example disposition, by location, of image capture events notified to 
the service system 14, the individual events being indicated by crosses. The events are 
those that have occurred within the locality of a person who has requested information 
about attractions and places of interest near to the person. As can be seen, most of the 
events lie within one or other of four groups 31 to 34. The clustering functionality 17A is 

1 5 operative to determine that there are four clusters, also referenced 3 1 to 34 and shown in 
Figure 3 by corresponding ovals. In Figure 3 the clusters are shown overlaid on a map of 
the locality showing what buildings are present. In responding to the request for general 
information about attractions and places of interest in the locality, the request handler 1 8 
will typically send back a combined cluster/map picture to the requestor (appropriate map 

20 data being retrieved from database 16 or any other suitable store). 

Figures 3 to 6 illustrate how the provision of direction of image capture data for each event 
of a cluster can assist in the derivation of further useful information about the cluster. The 
direction of image capture is derived by an appropriate sensor (such as a magnetic or 
25 electronic compass) associated with the camera functionality, the direction reading 
produced by the sensor being captured at the same time as the image capture event takes 
place and being included as a direction of capture parameter in the image-capture event 
notification 13 sent to the service system. 

30 Figure 3 depicts a group of seven events forming a cluster 40 as determined by the 
clustering functionality 1 7 A. By itself, knowledge of the form and location of a cluster is of 
some use to a requestor of information about attractions and places of interest in the 
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locality of the requestor. However, the addition of map features such as buildings 41 and 

42 increases the usefulness of the information supplied to the requestor. Nevertheless, this 
information does not indicate to the requestor which building is the one of interest - it 
could equally be building 41 or building 42. By using the direction of image capture 

5 information supplied with each event notification 13 the analysis functionality 17B is, 
however, able to determine that it is the building 41 that is the one of interest. In its 
simplest form the analysis carried out by functionality 1 7B is just to determine that more of 
the events are pointing towards building 4 1 than to 42 - indeed, as illustrated by the arrows 

43 coming from each event cross in the right-hand portion of Figure 4, all of the events 
1 0 concern image capture towards the building 4 1 . The analysis functionality 1 7B may further 

seek to pinpoint the likely subject of interest (such as an entranceway, a window etc. of 
building 41) by determining a point or area of intersection of the direction of capture lines 
(see dashed lines 44) with the building 41 or with whatever feature lies in the direction of 
the lines 44. The determined likely subject of the cluster of image capture events can then 
15 be reported to the requestor by highlighting on a map, by text or by any other suitable 
method. 

In the Figure 5 example, an image-capture event cluster 50 can be seen to lie in a courtyard 
formed by buildings 52 and in the center of which is a feature 5 1 . It is not clear from the 

20 form and location of the cluster as superimposed on the building map, whether it is the 
buildings 52 that are of interest or the feature 5 1 . However, when the direction of image 
capture for each event is taken into account (see arrows 53), it is clear that it is the feature 
51 that is of interest. The analysis functionality 17B can again readily determine that the 
feature 5 1 is the likely subject of interest, enabling the request handler to report this to any 

25 requestor. 

Identification of the subject of interest enables the processing sub-system 17 to 
automatically seek a source of information, and in particular a website or part of a website, 
about the subject of interest. To do this, the processing sub-system 17 can either use the 
30 location of the subject to look up a relevant website in a location-to-website index directly, 
or use a map name for the subject to look up a relevant website in a directory or search for 
one using a search engine. If a relevant website is found, the URL of the website can be 
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stored and provided by the request handler 1 8 to interested requestors. 



In the Figure 6 example, an image-capture event cluster 60 is found to lie in open country 

i 

and in this case, the cluster is overlaid on a contour map rather than a building map (see ? 
5 contour lines 6 1 that here represent a hill). With this example it is not clear whether or not I 
the majority of image capture events concern a feature on top of the hill. With the addition 

- 

of direction of image capture information (see arrows 63), it becomes clear that it is not the 
top of the hill that is of interest but the view in any direction from the hill. The analysis 
functionality 17B is able to detect this from the supplied direction of image capture 
10 information and, given appropriate rules, can determine that the cluster 60 is located on a 
viewpoint. This characteristic of the cluster location is stored and reported to requestors of 
information about the locality. Other characteristics concerning the locality of a cluster can 
be similarly deduced from the patterns of image-capture directions encountered. 

1 5 Figures 7 to 9 illustrate how knowing the time of occurrence of an image capture event can 
assist in the derivation of further useful information about a cluster of such events. A time 
indicative of when an image-capture event occurred can provided by one of: 

- a timestamp generated at the time of image capture by the camera functionality 10 
effecting the image-capture event, the timestamp being included in the image-capture 

20 event notification 13; 

- a timestamp added by the communications functionality 1 1 and indicative of the time 
of transmission of the image-capture event notification 13; 

- the time of receipt by the service system of the image-capture event notification. 
The latter two methods of providing time of occurrence information about an image- 

r- 

25 capture event are only useful where there is no major delay between the event itself and the 
sending/receipt of the corresponding event notification. 

In the Figure 7 example, the analysis functionality 17B is arranged to use the time of 
occurrence of new image capture events of a cluster to determine the current rate of 
30 addition of events to the cluster. More particularly, for each successive time period 70 to 
75 (for example, each of one minute duration) a count is made of the number of cluster 
events newly occurring. When a predetermined threshold level 77 is reached (at time 
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period 74 in Figure 7), the functionality 17B generates an alert to indicate that something 
interesting is probably happening. This alert can be sent by the request handler to all 
apparatuses 25 within a certain range of the cluster 25 (on the basis that users of the 
apparatuses have, by the very fact of using the service system 14, implicitly requested to be 
5 alerted to such happenings); alternatively, the alert can be sent by the request handler 18 
only to apparatuses that have explicitly requested to be notified of such alerts. Rather than 
the dissemination of alerts being limited to apparatuses within a predetermined range of the 
cluster concerned, a requestor can request to be notified of all alerts generated or all alerts 
generated within a specific geographic area. 

10 

The time period over which the rate of occurrence of new events is measured will depend 
upon various factors such as the nature of the subject of the image-capture events. If the 
intention is to be alerted to the occurrence of one-off attractions as they happen, then a 
short time period, typically ten minutes or less, is appropriate. However, for on-going 
1 5 attractions and places of interest, a requestor may only want to know when the attraction 
has reached a particular level of popularity and in this case a time period of a day may be 
appropriate. 

The rate of occurrence of new events in a cluster as a function of their time of occurrence 
20 effectively provides an image-capture event activity profile; such a profile is of interest 
independently of whether it is used to generate alerts. The analysis functionality 17B may 
therefore be arranged simply to determine and store this activity profile for supply by the 
request handler 18 to a requestor as needed. 

25 Figure 8 depicts the activity profile of an attraction that occurs daily. In Figure 8, three 
24hr periods 80, 81 and 82 are illustrated. The 24hr periodicity in the occurrence of events 
can be readily seen. However, the activity profile also contains additional information of 
interest. Thus, the activity profile for each day shows a double peak, one in the morning 
and one in the afternoon - the implication is that to avoid crowds it is better to arrive early 

30 or late, there only being a slight crowd reduction over the lunch period. Where the 
attraction that is the subject of the image-capture events has restricted access times, this 
will show up on the activity profile as strict start and stop times that do not correlate to 
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daylight hours or the start and stop of public transport. 

The analysis functionality 1 7B is preferably arranged to automatically detect periodicity in 
the occurrence of events of a cluster. This periodicity will typically be one (or more) of a 
5 daily periodicity, a weekly periodicity, a monthly periodicity, an annual periodicity. The 
periodicity of the attraction/place of interest associated with a cluster of image-capture 
events is preferably included to any report of the cluster to a requestor. The analysis 
functionality 1 7B can also arranged to detect any apparent artificial constraints on the time 
of occurrence of said image-capture events. 

10 

Figure 9 illustrates the generation of an alert on the basis described above with respect to 
Figure 7, namely that the rate of occurrence of new events in a cluster (as represented by 
the unit-time event count boxes 90) exceeds a predetermined threshold 91 . In the Figure 9 
example, the cluster concerned has been determined to have periodicity and the analysis 
1 5 functionality 1 7B has calculated an averaged activity profile (dashed line 92). This profile 
is reported along with the alert to any interested requestor in order to provide the requestor 
with an indication of the expected duration and popularity of the attraction associated with 
the event cluster. 

20 Where the image-capture events of a cluster that exhibits periodicity only occur within a 
narrow time window (or windows) of each cycle of the cluster periodicity, then it is 
advantageous to arrange for the analysis functionality 17B to detect this situation and to 
generate an alert to indicate an upcoming or just commencing time window predicted for a 
current cycle of the cluster periodicity independently of any event notifications received for 

25 that window. Such alerts can be treated in the same way as other alerts, being sent to all 
apparatuses 25 automatically or only to specific requestors, and with or without the 
application of filtering based either on the current location of the apparatuses or on a 
requestor-specified geographic area of interest. Furthermore, the alert is preferably 
accompanied by the activity profile for the time window to provide the recipient with an 

30 indication of the expected duration and popularity of the attraction concerned. 
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The analysis functionality 17B can also be arranged to use the location and time of 
occurrence data about new events to make successive determinations of a centre of 
accretion of events to a cluster. Where the centre of accretion is determined to be changing 
in a non-random manner, the analysis functionality can be arranged to infer movement of 
5 the subject of the image-capture events. Information about this movement can be reported 
by the request handler 18 to interested requestors. 

The service system is preferably arranged to discard data about events over a 
predetermined age at least for the purposes of the analyses conducted by the functionality 
10 17B. Alternatively, events over a predetermined age can given reduced weight as 
compared to other events of the same cluster, at least in respect of analyses of the cluster. 

In addition or alternatively to the analysis functionality 17B carrying out its analysis of a 
cluster on the basis of the time of occurrence and/or direction of image capture of image- 

1 5 capture events, the functionality can carry out analysis of a cluster on the basis of a source 
identifier associated with each event. Typically, this source identifier is either a user ID or 
an ID of the camera functionality 1 0 effecting the image-capture event concerned; in either 
case, the source identifier is generally included in the corresponding event notification. The 
analysis functionality 17B can be arranged to identify (and possibly count) events having 

20 the same source identifier in order to derive behaviour patterns or similar data. The 
identification of events having the same source identifier can also be used to identify the 
activity of persons, such as professional photographers, engaged to take photographs (for 
example, in a theme park). 

25 Other event-specific parameters that can be used as a basis for cluster analysis include the 
angle of elevation of image capture and camera settings such as focus distance, focal 
length, shutter speed, flash settings, etc. An indication of the focus distance is particularly 
useful in determining the subject of an image-capture event and can be used fro this 
purpose either alone or in conjunction with another parameter such as the direction of 

30 image capture. Furthermore, it may be useful to disregard image-capture events with very 
short focal distances that occur in open spaces as such events are likely to have a family or 
group member as there subject rather than a building or other more regular attraction. 
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It will be appreciated that many variants are possible to the above described embodiments 
of the invention. For example, as well as passing parameter data about an image capture 
event to the service system, it is also possible to arrange for the captured images to be sent 
to the service system for sharing with others. In this case, each cluster preferably has an 
associated image library which a requestor can reach by clicking on a representation 
(typically on a map) of the corresponding event cluster. Rather than providing images for 
everyone to view, images can be provided in encrypted form or for storage on protected 
sites accessible to a closed user group only. 

It is also possible to arrange for the request handler 18 to provide other information to a 
requestor such as information added manually at the service system about the subject of 
each event cluster, and information about how to get from the requestor's current location 
to the location of a cluster of interest to the requestor. 



